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DETAILED ACTION 

Response to Amendment 

1. This office action is in response to an amendment filed on 8/27/2008. 

2. Claims 1, 11, 12, 16, 19, 20, 25, 26 and 33 have been amended by the applicant. 

3. Claims 2-4, 6, 7, 10, 13-15, 17, 18, 21, 23, 24 and 34-36 are original. 

4. Claims 5, 8, 9, 22 and 27-32 have been cancelled. 

5. Claims 37-46 have been added. 



Claim Rejections - 35 USC §101 

35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

Claims 18 and 19 are rejected under 35 U.S.C. 101 as not falling within one of the 
four statutory categories of invention. While the claims recite a series of steps or acts to 
be performed, a statutory "process" under 35 U.S.C. 101 must (1) be tied to another 
statutory category (such as a particular apparatus), or (2) transform underlying subject 
matter (such as an article or material) to a different state or thing (Reference the May 15, 
2008 memorandum issued by Deputy Commissioner for Patent Examining Policy, John J. 
Love, titled "Clarification of 'Processes' under 35 U.S.C. 101"). The instant claims 
neither transform underlying subject matter nor positively tie to another statutory 
category that accomplishes the claimed method steps, and therefore do not qualify as a 
statutory process. For example, the image data reproduction method of claims 18 and 19 
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generate multimedia information files, however, the methods fail to tie the steps of the 
method to an apparatus or system in which to perform the method. 

Claim Rejections - 35 USC §103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 1-3, 6, 7, 12, 14-16, 20, 23, 24 and 33-46 are rejected under 35 U.S.C. 
103(a) as being unpatentable over Swift (US Patent 6,765,568) in view of Osaka et al. 
(hereinafter "Osaka", US Patent 6,023,277). 

Regarding claim 1, Swift teaches an image data generation apparatus (col. 5 lines 
21-23: "...computer 102... used to display... stereo image...") comprising: 

reception means for receiving a parameter for displaying three-dimensional image 
data (col. 3 lines 24-26 - 47-50: ". . .a. ..file format. ..of the original left and right is 
known, as designated by the tag within the. ..file. . ." and col. 7 lines 51-57: "The. ..system 
takes a Stereoscopic Media File... saves a local copy in. ..the display method selected. ..", 
where the system accepts a file and therefore contains a reception means to retrieve and 
save the file), including information data indicating one of a plurality of predetermined 
camera arrangements of a plurality of cameras which has picked up the three dimensional 
image data, said one of a plurality of camera arrangements describing a placement of said 
plurality of cameras relative to each other during image pickup (col. 9 lines 55-62: 
"...store a series of N images into a... file... the point of view of the camera is swung 



Application/Control Number: 10/512,058 Page 4 

Art Unit: 2628 

around the object to generate... the... images", where a file stores an object imaged from 
a plurality of relative camera view points that are initialized and saved by the user, col. 
10 lines 52-62: "Another embodiment allows ...views to be generated to simulate 
a... camera... The amount of adjustment can be selected by the user to suit their viewing 
condition and capabilities", therefore the file utilizes a camera parameter, in which the 
placement of the cameras in their relative positions is also provided, as illustrated in Fig 
19: 1908, 1906) and at least one of information data indicating a method of generating 
three-dimensional image data from data of the picked up image and information data for 
controlling presentation of the three-dimensional image data (col. 2 lines 28-41 : "The 
preferred embodiment... provides a single format with independent right and left 
channels... to represent the stereoscopic media... it provides a means of displaying 
stereoscopic media... it provides automatic and manual optimization adjustments such as 
parallax shift adjustment. ..to the stereoscopic media based on viewing hardware... for 
optimal viewing quality", where information is provided that controls how the three- 
dimensional image data is presented); 

three-dimensional image display control information generation means for 
generating three-dimensional image display control information by encoding said 
parameter (col. 4 lines 6-11: "The encoding processes used include independent 
compression of the Left and Right images. Independent compression of Left and Right 
provides better quality display output... methods include a video setup format for 
compression... Several tools are used... These include... a stereo compression analysis tool 
to detect if the stereo information is preserved after compression." and in col. 1 1 lines 
12-16: "If too much compression is applied, the image will loose its stereoscopic impact. 
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This embodiment provides a measure of the quality of the stereoscopic image that can be 
used to readjust the compression system", where a parameter, or stereo information, that 
provides information regarding how to correctly display and align the stereoscopic 
images form corresponding left/right points of view, is encoded in the file); and 

file generation means for generating a multimedia information file based on both 
of said three-dimensional image display control information (col. 3 lines 24-26 and 47- 
50: ". . .a single media file format that is converted to various display formats ...the format 
of the original left and right is ...designated by the tag within the... file. . .", Fig. 1) and 
said three-dimensional image data (col. 8 lines 1 1-20: "...a file structure is created to 
store and preserve various types of stereo media... This one file format can store multiple 
or single stereo/non-stereo media elements. FIG. 10 illustrates a VRR file 1000 that may 
contain ...a ...Stereoscopic Object Model...", where the stereoscopic media file also 
contains three-dimensional image data such as stereoscopic three-dimensional models). 

However, Swift fails to teach header control information added to a multimedia 
information file. Osaka teaches header control information (col. 5 lines 4-1 1 : 
". ..information indicating... a three-dimensional display. ..is provided in the header of a 
file... this information is used to decide execution of a three-dimensional display, thereby 
making it possible to readily decide whether a three-dimensional display is to be 
executed...", Fig. 12: element 51). Therefore it would have been obvious to one of 
ordinary skill in the art at the time of invention to combine the display control 
information of Swift with the file header data of Osaka because this combination would 
provide encoded three-dimensional image data stored in a file as taught by Swift, in 
which the three-dimensional data file comprises a header control data, as taught by 
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Osaka, where data processing of the three-dimensional image data is thereby reduced 
through analysis of the header control information which indicates the dimensional 
complexity of the objects within the file without requiring the entire file to be accessed. 

Regarding claim 2, Swift teaches a recording means for recording the multimedia 
file (col. 7 lines 51-54: "The viewing system takes a Stereoscopic Media File... and saves 
a local copy... "). 

Regarding claim 3, Swift teaches the file generation means outputs said 
multimedia information file to an external communication path (col. 7 lines 49-54: 
"There is an embodiment that saves and converts one format into another from the 
Internet using a local drive from original source. The viewing system takes a 
Stereoscopic Media File, displays it on the user side according to the user's display 
preferences, and saves a local copy..."). 

Regarding claims 6 and 23, Swift teaches providing a particular file format (col. 3 
lines 23-26: ". . .the invention... includes a. ..media file format. .."), however Swift fails to 
teach indicating a file type by providing a different extension for the file when it is 
determined whether or not the file contains three-dimensional image data. Osaka teaches 
a file generation means indicates a file type by providing a different extension to said 
multimedia information file when said multimedia information file contains the three- 
dimensional image data and when said multimedia information file does not contain 
three-dimensional image data (col. 5 lines 4-11: "...to provide a display control 
apparatus and a display control method in which paint information indicating whether a 
three-dimensional display is . . .provided in the header of a file ..."). The motivation to 
combine the teachings of Swift with Osaka is equivalent to the motivation of claim 1 . 
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Regarding claims 7 and 24, Swift fails to teach the limitations. Osaka teaches that 
when said extension indicates that said multimedia information file contains three- 
dimensional image data, the extension further indicates which of the plurality of different 
three-dimensional display methods, with each display method being different from the 
other, is used to display said three-dimensional image data (col. 17 lines 26-41 : 
". . .whether or not a three-dimensional image is to be displayed has been recorded as 
information in the header 51 and therefore the decision of step S64 is rendered promptly 
by making reference to this information... these processing operations may be executed by 
a file extension... If it is determined at step S64 that this window has a three-dimensional 
image file... the display driver 6 is controlled to present the three-dimensional 
display... on the basis of the three-dimensional image data obtained from the three- 
dimensional image data 52, the screen controller 9 controls the image painting unit 7 
and the checkered mask-pattern painting unit 8 and causes a three-dimensional display 
to be presented at the position of the window of the stereoscopic display 12"). The 
motivation to combine the teachings of Swift with Osaka is equivalent to the motivation 
of claim 1 . 

Regarding claim 12, Swift teaches an image data reproduction apparatus (col. 5 
lines 21-23: "...computer 102.. .used to display... stereo image...", Fig. 2: element 102) 
comprising: 

reception means for receiving a multimedia information file including both of 
three-dimensional image display control information (col. 2 lines 28-41: "The preferred 
embodiment... provides a single format with independent right and left channels... to 
represent the stereoscopic media... it provides a means of displaying stereoscopic 
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media... it provides automatic and manual optimization adjustments such as parallax shift 
adjustment... to the stereoscopic media based on viewing hardware... for optimal viewing 
quality."') and said three-dimensional image data (col. 8 lines 1 1-20: "An embodiment 
supports a stereoscopic media file that contains sub-media. Specifically, a file structure is 
created to store and preserve various types of stereo media... This one file format can 
store multiple or single stereo/non-stereo media elements. FIG. 10 illustrates a VRR file 
1000 that may contain... a Stereo Still Image 1004. ..Stereoscopic Object Model. 
where the stereoscopic media file also contains three-dimensional image data), generated 
by encoding a parameter for displaying three-dimensional image data and said three- 
dimensional image data, or two-dimensional image data (col. 4 lines 6-11: "The encoding 
processes used include independent compression of the Left and Right images. 
Independent compression of Left and Right provides better quality display 
output... methods include a video setup format for compression... Several tools are used in 
the process. These include. ..a stereo compression analysis tool to detect if the stereo 
information is preserved after compression." and in col. 1 1 lines 12-16: "If too much 
compression is applied, the image will loose its stereoscopic impact. This embodiment 
provides a measure of the quality of the stereoscopic image that can be used to readjust 
the compression system."); 

wherein the parameter including information data indicating one of a plurality of 
predetermined camera arrangements of a plurality of cameras which has picked up the 
three dimensional image data, said one of a plurality of camera arrangements describing a 
placement of said plurality of cameras relative to each other during image pickup (col. 9 
lines 55-62: "...store a series of N images into a... file... the point of view of the camera is 
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swung around the object to generate... the... images", where a file stores an object 
imaged from a plurality of relative camera view points that are initialized and saved by 
the user, col. 10 lines 52-62: "Another embodiment allows ...views to be generated to 
simulate a... camera... The amount of adjustment can be selected by the user to suit their 
viewing condition and capabilities", therefore the file utilizes a camera parameter, in 
which the placement of the cameras in their relative positions is also provided, as 
illustrated in Fig 19: 1908, 1906), and information data indicating a method of generating 
three-dimensional image data from data of the picked up image and information data for 
controlling presentation of the said three-dimensional image data (col. 9 lines 53-62: 
"One embodiment stores a series of 3D stereoscopic images of an object 
into... a... file... the point of view of the camera is swung around the object to generate all 
of the intermediate images." and in col. 2 lines 28-41: "The preferred 
embodiment... provides a single format. ..to represent the stereoscopic media. ..it provides 
a means of displaying stereoscopic media... it provides automatic and manual 
optimization adjustments such as parallax shift adjustment. ..to the stereoscopic media 
based on viewing hardware... for optimal viewing quality."); 

file structure analysis means for analyzing a structure of said multimedia 
information file, such as the file format, so as to extract the three-dimensional image 
display control information and the three-dimensional image data (col. 3 lines 23-50: 
"...the invention... includes a. ..media file format... to... display formats on the user 
side... support of auto-detection 3D stereo hardware systems... automatic... calibration 
/adjustments for physical 3D viewing mechanisms... and stereoscopic viewing that... as 
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designated by the tag within the Stereoscopic 3D Media file.. .", where the file format is 
analyzed to determine the associated display format); 

three-dimensional image display control information analysis means for analyzing 
said three-dimensional image display control information extracted by the file structure 
analysis and data reproduction means for reproducing said three-dimensional image data 
extracted by the file structure analysis (col. 3 lines 24-50: ". ..it provides a single format 
with independent right and left channels... to represent the stereoscopic media... it 
provides automatic and manual optimization adjustments such as parallax shift 
adjustment, brightness control, color adjustment, and cross-talk reduction to the 
stereoscopic media...", in which the file format is analyzed to determine the display 
control data); 

data conversion means for converting reproduced three-dimensional image data, 
such that the three-dimensional image data can be presented in a display unit, based on a 
result of analysis by the three-dimensional image display control information analysis 
(col. 3 lines 24-26: ". . .a single media file format that is converted to various display 
formats on the user side..." and in col. 8 lines 45-63: "Another embodiment allows for 
automatic detection of 3D display hardware... an appropriate display method can be 
automatically selected... Another embodiment automatically (or manually if desired) 
adjusts with overall left and right image shift... the 3D stereoscopic image is adjusted 
accordingly. . .", wherein based on the display device, the file format of the display 
control information is analyzed and is converted to the proper display control file format 
required to accurately display the three-dimensional data, thereby correctly reproducing 
the 3D media on the display ): 
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However, Swift fails to teach header control information. Osaka teaches header 
control information (col. 5 lines 4-11: ". ..information indicating whether a three- 
dimensional display is possible... provided in the header of a file... this information is 
used to decide execution of a three-dimensional display, thereby making it possible to 
readily decide whether a three-dimensional display is to be executed../'), where the 
display of the three-dimensional data produced with respect to the header information 
that is analyzed from the file (Fig. 12), in which one of ordinary skill in the art would 
have modified the teachings of Swift with Osaka because the three-dimensional image 
data comprised within the file disclosed by Swift (col. 8 lines 1 1-20: "...a stereoscopic 
media file... a file structure is created to store and preserve various types of stereo 
media. ..This one file format can store... Stereoscopic Object Model. .."), would have been 
correctly analyzed through utilizing the information contained within the header control 
information disclosed by Osaka (col. 5 lines 4-11: "...information indicating... a three- 
dimensional display is... provided in the header of a file... this information is used to 
decide execution of a three-dimensional display, thereby making it possible to readily 
decide whether a three-dimensional display is to be executed...") due to the efficient 
retrieval of three-dimensional display control analysis provided by the access of the 
header control information that corresponds to the file. Therefore it would have been 
obvious to one of ordinary skill in the art at the time of invention to modify the three- 
dimensional data file of Swift with the file header information of Osaka because this 
modification would provide the ability to display stereoscopic images stored in a 
particular display format on any display device, as taught by Swift, in which the display 
control processing is reduced through the indication of the presence of three-dimensional 
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data within the file, as taught by Osaka, thereby quickly identifying and distinguishing 
whether two or three-dimensional display control is required during display. 

Regarding claims 14 and 15, Swift providing a particular file format that indicates 
3D media (col. 3 lines 24-26 and 47-50: "...a single media file format that is converted to 
various display formats... the format of the original left and right is... designated by the 
tag within the... file...", Fig. 1), however Swift fails to teach analyzing a structure of the 
multimedia information file to determine whether three-dimensional image data is 
contained within the file. Osaka teaches a file type determination means for analyzing a 
structure of said multimedia information file so as to determine whether the multimedia 
information file includes at least three-dimensional image display control information 
(col. 17 lines 26-41 : " ...whether or not a three-dimensional image is to be displayed has 
been recorded as information in the header 51 and therefore the decision of step S64 is 
rendered promptly by making reference to this information... these processing operations 
may be executed by a file extension... If it is determined at step S64 that this window has a 
three-dimensional image file... the display driver 6 is controlled to present the three- 
dimensional display... on the basis of the three-dimensional image data obtained from the 
three-dimensional image data 52, the screen controller 9 controls the image painting unit 
7 and the checkered mask-pattern painting unit 8 and causes a three-dimensional display 
to be presented at the position of the window of the stereoscopic display 12"). The 
motivation to combine the teachings of Swift with Osaka is equivalent to the motivation 
of claim 1 . 



Application/Control Number: 10/512,058 Page 
Art Unit: 2628 

Regarding claim 16, Swift teaches an image data reproduction apparatus (col. 5 
lines 21-23: ". . .computer 102.. .used to display ...stereo image...", Fig. 2: element 102) 
comprising: 

reception means for receiving a multimedia information file including three- 
dimensional image display control information (col. 3 lines 24-26 - 47-50: "...a. ..file 
format... of the original left and right is known, as designated by the tag within 
the... file. . .") obtained by encoding a parameter for displaying three-dimensional image 
data and said three-dimensional image data, or two-dimensional image data (col. 4 lines 
6-11: "The encoding processes used include independent compression of the Left and 
Right images. Independent compression of Left and Right provides better quality display 
output... methods include a video setup format for compression..." and col. 2 lines 29-31: 
". . .delivering stereoscopic media in electronic form. ..object models, etc. . ."), a parameter 
including information data indicating one of a plurality of predetermined camera 
arrangements of a plurality of cameras which has picked up the three dimensional image 
data, said one of a plurality of camera arrangements describing a placement of said 
plurality of cameras relative to each other during image pickup (col. 9 lines 55-62: 
". . .store a series of N images into a... file... the point of view of the camera is swung 
around the object to generate... the... images", where a file stores an object imaged from 
a plurality of relative camera view points that are initialized and saved by the user, col. 
10 lines 52-62: "Another embodiment allows ...views to be generated to simulate 
a... camera... The amount of adjustment can be selected by the user to suit their viewing 
condition and capabilities", therefore the file utilizes a camera parameter, in which the 
placement of the cameras in their relative positions is also provided, as illustrated in Fig 
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19: 1908, 1906), and at least one of information data indicating a method of generating 

three-dimensional image data from data of the picked up image and information data for 

controlling presentation of the said three-dimensional image data (col. 9 lines 53-62: 

"One embodiment stores a series of 3D stereoscopic images of an object 

into... a... file... the point of view of the camera is swung around the object to generate all 

of the intermediate images." and in col. 2 lines 28-41: "The preferred 

embodiment... provides a single format. ..to represent the stereoscopic media. ..it provides 

a means of displaying stereoscopic media... it provides automatic and manual 

optimization adjustments such as parallax shijt adjustment... to the stereoscopic media 

based on viewing hardware... for optimal viewing quality."); 

However, Swift fails to teach file type determination. Osaka teaches a file type 
determination means for analyzing an extension of said multimedia information file 
where the file type determination means determines, based on the analysis of the 
extension, whether said multimedia information file includes said three-dimensional 
image data and determines on which of a plurality of three dimensional display schemes 
is based (col. 17 lines 23-41 : ". ..it is determined, based upon the information in the file 
header 51, whether this window has three-dimensional image data. In this example, 
whether or not a three-dimensional image is to be displayed has been recorded as 
information in the header 51 and therefore the decision of step S64 is rendered promptly 
by making reference to this information... these processing operations may be executed by 
a file extension... If it is determined at step S64 that this window has a three-dimensional 
image file... the display driver 6 is controlled to present the three-dimensional 
display ...on the basis of the three-dimensional image data obtained from the three- 
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dimensional image data 52, the screen controller 9 controls the image painting unit 7 
and the checkered mask-pattern painting unit 8 and causes a three-dimensional display 
to be presented at the position of the window of the stereoscopic display 12."). Therefore 
it would have been obvious to one of ordinary skill in the art at the time of invention to 
modify the three-dimensional data file of Swift with the file header information of Osaka 
because this modification would provide the ability to display stereoscopic images stored 
in a particular display format on any display device, as taught by Swift, in which the 
display control processing is reduced through the indication of the presence of three- 
dimensional data within the file, as taught by Osaka, thereby quickly identifying and 
distinguishing whether two or three-dimensional display control is required during 
display. 

Regarding claim 20, Swift teaches an image data recording medium having stored 
thereon information representing a multimedia information file, where the information 
executed on an image data reproduction apparatus (col. 12 lines 6-17: "...invention can 
also be embodied in the form of computer program code, for example... stored in a 
storage medium, loaded into and/or executed by a computer ...wherein, when the 
computer program code is loaded into and executed by a computer, the computer 
becomes an apparatus for practicing the invention.'''') depicting information including 
both of three-dimensional image display control information generated by encoding a 
parameter for displaying three-dimensional image data (col. 4 lines 6-11: "The encoding 
processes used include... compression of the Left and Right images."); 

wherein the parameter includes information data indicating a camera arrangement 
that has picked up said three-dimensional image data (col. 9 lines 55-62: ". . .store a 
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series of N images into a... file... the point of view of the camera is swung around the 
object to generate... the... images." and in col. 10 lines 52-62: "Another embodiment 
allows... views to be generated to simulate a... camera... The amount of adjustment can be 
selected by the user to suit their viewing condition and capabilities", where a file stores 
camera data from each given view used to generated the images), and information data 
indicating a method of generating three-dimensional image data from data of the picked 
up image and information data for controlling presentation of the said three-dimensional 
image data (col. 9 lines 53-62: "One embodiment stores a series of 3D stereoscopic 
images of an object into... a... file... the point of view of the camera is swung around the 
object to generate all of the intermediate images." and in col. 2 lines 28-41 : "The 
preferred embodiment... provides a single format. ..to represent the stereoscopic 
media... it provides a means of displaying stereoscopic media... it provides automatic and 
manual optimization adjustments such as parallax shift adjustment... to the stereoscopic 
media based on viewing hardware... for optimal viewing quality"). 

However, Swift fails to teach header control information. Osaka teaches header 
control information (col. 5 lines 4-11: ". . .information indicating whether a three- 
dimensional display is possible or not is provided in the header of a file... this information 
is used to decide execution of a three-dimensional display, thereby making it possible to 
readily decide whether a three-dimensional display is to be executed or not", Fig. 12: 
element 51). Therefore it would have been obvious to one of ordinary skill in the art at 
the time of invention to combine the display control information of Swift with the file 
header data of Osaka because this combination would provide encoded three-dimensional 
image data as taught by Swift, in which the three-dimensional data file comprises a 
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header control data, as taught by Osaka, in which the data processing is reduced through 
analysis of the header control information which indicates what types of objects the file 
contains without requiring the entire file to be processed. 

Regarding claim 33, Swift teaches an image data recording medium having stored 
thereon information representing a multimedia information file, where the information 
executed on an image data reproduction apparatus (col. 12 lines 6-17: ". . .invention can 
also be embodied in the form of computer program code, for example... stored in a 
storage medium, loaded into and/or executed by a computer ...wherein, when the 
computer program code is loaded into and executed by a computer, the computer 
becomes an apparatus for practicing the invention"), depicting information including 
three-dimensional image display control information (col. 3 lines 24-26 - 47-50: 
". . .a. ..file format. ..of the original left and right is known, as designated by the tag within 
the... file. . ."), generated by encoding a parameter for displaying three-dimensional image 
data (col. 4 lines 6-11: "The encoding processes used include independent compression of 
the Left and Right images. Independent compression of Left and Right provides better 
quality display output... methods include a video setup format for compression ...Several 
tools are used in the process. These include... a stereo compression analysis tool to detect 
if the stereo information is preserved after compression.' 1 '') and said three-dimensional 
image data (col. 1 1 lines 12-16: "If too much compression is applied, the image will loose 
its stereoscopic impact. This embodiment provides a measure of the quality of the 
stereoscopic image that can be used to readjust the compression system."); 

wherein the parameter including information data indicating one of a plurality of 
predetermined camera arrangements of a plurality of cameras which has picked up the 
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three dimensional image data, said one of a plurality of camera arrangements describing a 
placement of said plurality of cameras relative to each other during image pickup (col. 9 
lines 55-62: "...store a series of N images into a... file... the point of view of the camera is 
swung around the object to generate... the... images", where a file stores an object 
imaged from a plurality of relative camera view points that are initialized and saved by 
the user, col. 10 lines 52-62: "Another embodiment allows ...views to be generated to 
simulate a... camera... The amount of adjustment can be selected by the user to suit their 
viewing condition and capabilities", therefore the file utilizes a camera parameter, in 
which the placement of the cameras in their relative positions is also provided, as 
illustrated in Fig 19: 1908, 1906), and information data indicating a method of generating 
three-dimensional image data from data of the picked up image and information data for 
controlling presentation of the said three-dimensional image data (col. 9 lines 53-62: 
"One embodiment stores a series of 3D stereoscopic images of an object 
into... a... file... the point of view of the camera is swung around the object to generate all 
of the intermediate images." and in col. 2 lines 28-41 : "The preferred 
embodiment... provides a single format. ..to represent the stereoscopic media. ..it provides 
a means of displaying stereoscopic media... it provides automatic and manual 
optimization adjustments such as parallax shift adjustment... to the stereoscopic media 
based on viewing hardware... for optimal viewing quality."); 

the recording area further including an image recording area for recording three- 
dimensional image data (col. 2 lines 21-25: "This invention presents new stereoscopic 
media delivery system that includes means for ...storing stereoscopic media...", Fig. 1 : 
element 12), an audio recording area for recording audio data (col. 8 lines 10-23: "...a 
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file structure is created to store and preserve various types of stereo media in various 
formats... this file format can also store... audio..."), and a sub code area for recording 
associated information, which has been interpreted to be any fragment or portion of data 
storage in the file containing data such as video or audio (col. 8 lines 10-23: "A stereo 
media file format may contain certain sub media. ..An embodiment supports a 
stereoscopic media file that contains sub-media... this file format can also store 
monoscopic media, as well as audio or other data. This one file format can store 
multiple or single stereo/non-stereo media elements. FIG. 10 illustrates a VRR file 1000 
that may contain a script 1002, a Stereo Still Image 1004, a Stereoscopic 
Animation/movie 1006, Stereoscopic Object Model 1008, a Thumbnail 1010, and Audio 
1012."); 

However, Swift fails to teach a recording area comprising information regarding a 
time code. Osaka teaches storing information regarding a time code for the image data 
(col. 20 lines 61-67 - col. 21 lines 1-5: ".. .image data is read out of the. ..image 
file... images are displayed continuously several viewpoints at a time... by the crossed- 
lenticular scheme..." and in col. 13 lines 16-18: "...images are displayed in time-division 
fashion...", where images are continuously displayed at particular times, therefore the 
system implicitly stores time instances or time codes related to teach image within a 
portion of the data memory storage space in order to correctly provide the images to the 
user). Therefore it would have been obvious to one of ordinary skill in the art at the time 
of invention to modify the three-dimensional image data file taught by Swift with the sub 
code time information taught by Osaka because this modification would provide encoded 
three-dimensional image data as taught by Swift, in which the three-dimensional data file 
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comprises time data, as taught by Osaka, where image data is improved through 
synchronization of time instances for the displayed images thereby preserving the three- 
dimensional data. 

Regarding claim 34, Swift illustrates information that when executed, records at 
least a portion of said three-dimensional image display control information in the image 
recording area (Fig. 1), where a portion of the display control information (14, 16, 18, 20, 
22, 24, 26) is recorded in the media file (12). 

Regarding claim 35, Swift teaches information that when executed, records at 
least a portion of said three-dimensional image display control information in the audio 
recording area (col. 8 lines 10-23: "A stereo media file format may contain certain sub 
media... An embodiment supports a stereoscopic media file that contains sub-media... this 
file format can also store monoscopic media, as well as audio or other data"), where any 
media data within the stereo file (Fig. 1 : element 12), may be stored with the audio 
therefore the audio file may contain video or display control information. 

Regarding claim 36, Swift teaches information that when executed, records at 
least a portion of said three-dimensional image display control information in the sub 
code area (col. 8 lines 10-23: "A stereo media file format may contain certain sub 
media... An embodiment supports a stereoscopic media file that contains sub-media... this 
file format can also store monoscopic media, as well as audio or other data. This one file 
format can store multiple or single stereo/non-stereo media elements. FIG. 10 illustrates 
a VRR file 1000 that may contain a script 1002, a Stereo Still Image 1004, a Stereoscopic 
Animation/movie 1006, Stereoscopic Object Model 1008, a Thumbnail 1010, and Audio 
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1012"), where any media data may be stored with the sub media, therefore the sub media 
file may contain video or display control information. 

Regarding claims 37, 39, 41, 43 and 45, Swift teaches information data 
indicating one of a plurality of camera arrangements describes at least one of a spatial 
relation between the plurality of cameras that has picked up said three-dimensional image 
data (col. 10 lines 62-63: ". . .right 1902 and left 1904 camera views of an object 1900." 
and is shown in Fig. 19: 1906, 1908). 

Regarding claims 38, 40, 42, 44 and 46, Swift teaches that the information data 
indicating one of a plurality of camera arrangements is one of an annular arrangement 
(Fig. 19: 1906, 1908, wherein the cameras are arrangement in a curved annular 
manner). 

Claims 4, 13, 17 and 21 rejected under 35 U.S.C. 103(a) as being unpatentable 
over Swift in view of Osaka, in further view of Connell (US Patent 5,581,625) and in 
further view of Harman (US Patent 6,496,598). 

Regarding claims 4, 13, 17 and 21, Swift teaches three-dimensional image display 
control information (Fig. 1 : element 10), including information indicating a number of 
viewpoints of the three-dimensional image data and information indicating from which 
viewpoint position the three-dimensional image data is obtained (col. 9 lines 53-62: "One 
embodiment stores a series of 3D stereoscopic images of an object into one file. FIG. 16 
illustrates one way to store a series of N images into a single resource file. The first 
image 1600 is at the top of the file and the rest of the images follow sequentially until the 
last file 1602... the point of view of the camera is swung around the object to generate all 
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of the intermediate images", Fig. 17). Swift also teaches indicating a maximum shift 
amount when a parallax image of three-dimensional data is shifted (col. 8 lines 51-63: 
". . .automatically (or manually if desired) adjusts with overall left and right image shift 
to compensate for image magn ification. When a 3D stereoscopic image is 
enlarged... there is a potential of creating large separations between objects in the 3D 
stereoscopic image that can lead to eyestrain for the viewer. The present invention stores 
important parameters about the 3D stereoscopic image like width, height, target screen 
size, etc. When the 3D stereoscopic image is to be displayed on a display that is larger or 
smaller than the target screen size, then the 3D stereoscopic image is adjusted 
accordingly to minimize eye fatigue for the user"), where the shift of the image is 
performed until it reaches a position in which if shifted further, the three-dimensional 
depth would not be preserved. However, Swift fails to teach identification of three- 
dimensional data within a multimedia information file. Osaka teaches identification of 
three-dimensional data within a multimedia information file (col. 17 lines 23-26: ". ..it is 
determined, based upon the information in the file header 51, whether this window has 
three-dimensional image data..."). Swift and Osaka fail to teach a direction of sub- 
sampling three-dimensional data. Connell teaches a direction of sub-sampling three- 
dimensional data (col. 7 lines 18-25: "The combined stereo video signal is then fed to 
computer 52, which digitizes same, separates the left and right images and subsamples 
each to obtain two compressed ...images (100 pixels across and 65 up and down). Prior 
to subsampling or compression, the combined image is 512 by 512 pixels (each half 
image is 512 pixels across and 256 up and down). Compression extracts, for example, 
every fifth pixel across and up and down. . ."). However, Swift, Osaka and Connell fail to 



Application/Control Number: 10/512,058 Page 
Art Unit: 2628 

teach indicating whether a border is to be display around and image, and an identification 
of three-dimensional data within a multimedia information file. Harman teaches 
indicating whether a border is to be displayed around an image of said three-dimensional 
image data (col. 6 lines 45-51 : "The processes would then work in either real-time or non 
real-time in order to create the 3D images. This can be further optimised through the use 
of borders... The 3D images may then be stored or transmitted to a 3D display, including 
shutterglasses, polarising glasses or an autostereoscopic display"), and indicating 
border image data to be displayed around the image of said three-dimensional image data 
(col. 14 lines 4-20: "The present invention therefore preferably also defines a common 
border or reference point within a viewed image. ..This has the advantage of enhancing 
the stereoscopic effect in many scenes. This reference point can be a simple video border 
or...i) A simple coloured video border around the perimeter of the image, ii) A complex 
coloured video border consisting of two or more concentric borders. . ."). Therefore it 
would have been obvious to one of ordinary skill in the art at the time of invention to 
combine the stored three-dimensional data of Swift, with the file header data of Osaka 
and sub-sampling display properties of Connell, with the optimized three-dimensional 
border display parameters of Harman because this combination would provide an 
accurate display of stereoscopic image data that has been analyzed to determine the three- 
dimensional data to be displayed, in which adjustment through shifting and modifications 
to the image data using borders based on the determined display information enables 
preservation of the stereoscopic effect. 
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Claims 10, 18 and 25 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Swift in view of Osaka in further view of Tanaka (US Patent 6,233,004). 

Regarding claims 10, 18 and 25, Swift teaches an image data generation apparatus 
(col. 5 lines 21-23: "...computer 102. ..used to display... stereo image...") comprising: 

a file generation unit for generating a multimedia information file including both 
of image pick-up condition information indicating an image pick-up condition for a three- 
dimensional image (col. 10 lines 53-54: ". ..embodiment allows new right and left image 
views to be generated to simulate ...stereoscopic camera separation", therefore the 
apparatus contains one or more equivalent generation units that would execute the 
method of generating the images stored on a computer-readable medium), a multimedia 
information file that contains three-dimensional image data (col. 2 lines 29-3 1 : 
". . .delivering stereoscopic media in electronic form. ..object models, etc. . ."); 

wherein both the pick-up condition, which provides information regarding image 
data related to images for the left and right eyes, and three-dimensional image data are 
encoded in a parameter stored in the file (col. 9 lines 53-62: "One embodiment stores a 
series of 3D stereoscopic images of an object into one file... the images follow 
sequentially until the last file 1 602... the point of view of the camera is swung around the 
object to generate... the... images." and in col. 4 lines 6-1 1 : "The encoding processes used 
include independent compression of the Left and Right images. Independent compression 
of Left and Right provides better quality display output... methods include a video setup 
format for compression... Several tools are used... These include... a stereo compression 
analysis tool to detect if the stereo information is preserved after compression.") . 
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However, Swift fails to teach header control information and image pick-up 
information including information indicating a number of parallaxes in a horizontal 
direction and perpendicular thereto. Osaka teaches header control information (col. 5 
lines 4-1 1 : ". . .to provide a display control apparatus and a display control method in 
which paint information indicating whether a three-dimensional display is possible or not 
is provided in the header of a file.. .", Fig. 12: element 51). 

However, Swift and Osaka fail to teach image pick-up information including 
information indicating a number of parallaxes in a horizontal direction and perpendicular 
thereto. Tanaka teaches image pick-up information indicating a number of parallaxes, or 
viewpoints, in a horizontal direction and a direction perpendicular thereto, which is 
therefore in a vertical direction (col. 4 lines 26-3 1 and 45-47: ". ..the aforesaid plurality 
of cameras 22 pick up images of the object 21, and the plurality of cameras... are 
arranged in both the vertical and horizontal directions on a plane apart from the object 
21 at a predetermined distance. . ."). Therefore it would have been obvious to one of 
ordinary skill in the art at the time of invention to combine the three-dimensional data of 
Swift and file header display control information of Osaka with the parallax display 
properties of Tanaka because this combination would provide accurate display of 
stereoscopic images through the analysis of the placement of several viewpoints required 
to display three-dimensional images correctly thereby preserving the depth and reducing 
visual discomfort experienced while viewing the images. 
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Claims 11,19 and 26 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Swift in view of Osaka in further view of Uomori et al. (hereinafter "Uomori", US 
Patent 6,005,607). 

Regarding claims 11,19 and 26, Swift teaches an image data generation apparatus 
(col. 5 lines 21-23: "...computer 102. ..used to display ...stereo image...") comprising: 

a file generation unit for generating a multimedia information file including both 
of image pick-up condition information indicating an image pick-up condition for a three- 
dimensional image (col. 9 lines 53-62: "One embodiment stores a series of 3D 
stereoscopic images of an object into one file... to store a series of ~N images into a single 
resource file. The first image 1600 is at the top of the file and the rest of the images 
follow sequentially until the last file 1602... the point of view of the camera is swung 
around the object to generate all of the intermediate images." and in col. 10 lines 52-65: 
". . .embodiment allows ...image views to be generated to simulate a... camera...", 
therefore the apparatus contains one or more equivalent generation units that would 
execute the method of generating the images stored on a computer-readable medium), 
and also teaches a file that contains three-dimensional image data (col. 2 lines 29-3 1 : 
". . .delivering stereoscopic media in electronic form. ..object models, etc. . ."); 

wherein both the pick-up condition, which provides information regarding image 
data related to images for the left and right eyes, and three-dimensional image data are 
encoded in a parameter stored in the file (col. 9 lines 53-62: "One embodiment stores a 
series of 3D stereoscopic images of an object into one file... the images follow 
sequentially until the last file 1602... the point of view of the camera is swung around the 
object to generate... the... images." and in col. 4 lines 6-1 1 : "The encoding processes used 
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include independent compression of the Left and Right images. Independent compression 
of Left and Right provides better quality display output... methods include a video setup 
format for compression... Several tools are used... These include... a stereo compression 
analysis tool to detect if the stereo information is preserved after compression"), and 

wherein said image pick-up condition information includes at least one of 
information indicating one of a plurality of predetermined camera arrangement shapes 
formed by a plurality of cameras which has picked up said three-dimensional image data 
(col. 9 lines 55-62: ". . .store a series of N images into a... file... the point of view of the 
camera is swung around the object to generate... the... images", col. 10 lines 52-62: 
"Another embodiment allows... views to be generated to simulate a... camera... The 
amount of adjustment can be selected by the user to suit their viewing condition and 
capabilities", as illustrated in Fig 19: 1906, 1908, in which a plurality of predetermined 
camera arrangements are provided for acquiring three-dimensional image data from the 
plurality of cameras), information indicating a camera arrangement shape indicating an 
interval between adjacent cameras of the plurality of cameras which has picked up said 
three-dimensional image data (col. 10 lines 52-65: ".. .embodiment allows ...image views 
to be generated to simulate a. ..camera. ..and. ..can be used to simulate a wider 
stereoscopic camera...", Fig. 19). 

However, Swift fails to teach header control information and information 
indicating a distance from a camera arrangement plane formed by the plurality of 
cameras which has picked up said three-dimensional image data to a convergence point 
formed by the view axes of said plurality of cameras. Osaka teaches header control 
information (col. 5 lines 4-1 1 : ". . Jo provide a display control apparatus and a display 
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control method in which paint information indicating whether a three-dimensional 
display is possible or not is provided in the header of a file. . .", Fig. 12: element 51). 

However, Swift and Osaka fail to teach indicating a distance from a camera 
arrangement plane formed by the plurality of cameras which has picked up said three- 
dimensional image data to a convergence point formed by the view axes of said plurality 
of cameras. Uomori teaches indicating a distance from a camera arrangement plane 
formed by the plurality of cameras which has picked up said three-dimensional image 
data to a convergence point formed by the view axes of said plurality of cameras, which 
is comprised in camera parameter information (col. 5 lines 24-3 1 : ". ..the fusional range 
verification section 11 calculates... the camera parameters ...distance dx from camera 
converging point to imaginary camera position V...", and is shown in Fig. 2: dx, where 
the view axes of the plurality of camera positions converge to the convergence point), 
therefore it would have been obvious to one of ordinary skill in the art at the time of 
invention to modify the three-dimensional data of Swift and file header display control 
information of Osaka, with the parallax camera arrangement properties of Uomori 
because this modification would provide visually appealing stereoscopic images through 
accurate adjustment of the parallax and viewpoints of the image acquisition devices with 
respect to calculated distances from the device to a convergence point. 

Response to Arguments 

Applicant's arguments with respect to claims 1-4, 6, 7, 10-21, 23-26 and 33-46 
have been considered but are moot in view of the new ground(s) of rejection. 
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The objection to the Specification has been withdrawn due to the amendments to 
the tile of the application. 

The 35 U.S.C. 101 rejection of claims 20, 21, 23-26 and 33-36 has been 
withdrawn due to the amendments to the specification to remove reference to the 
"transmission media". 

The applicant argues on pg. 18 3 rd H line 1 - 4 th K line 2 and on pg. 20 1 st If lines 1- 
6 of the remarks that Swift nor Osaka teach the limitations of claims 1 , however Swift 
and Osaka's teaching of the limitations of claim 1 are provided in the above office action, 
therefore the 35 U.S.C. 103(a) rejection of claim 1 has been maintained. 

The applicant argues on pg. 19 2 nd ^ lines 3-12 and in the 4 th U lines 1-3 of the 
remarks that Swift teaches a "tag" for displaying three-dimensional image data and 
cannot indicate one of a plurality of predetermined camera arrangements of a plurality of 
camera which has picked up said three-dimensional image data, in which no mention of 
the arrangement of the cameras is to be found in Swift. However, Swift provides teaching 
of the a plurality of predetermined camera arrangements of a plurality of camera which 
has picked up said three-dimensional image data (col. 10 lines 62-63: ". . .right 1902 and 
left 1904 camera views of an object 1900." and is shown in Fig. 19: 1906, 1908), where 
the arrangement of a plurality of cameras, as shown in Fig. 19, is thereby provided. 

The applicant argues on pg. 19 3 rd U lines 2-9 of the remarks that a camera 
arrangement is not expressly disclosed by Swift, and that the presence of a right and left 
channel is not a parameter including information data indicating one of a plurality of 
predetermined camera arrangements. However, Swift provides arrangement of a plurality 
of cameras to acquire image data (col. 10 lines 62-63: ". . .right 1902 and left 1904 
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camera views of an object 1900." and is shown in Fig. 19: 1906, 1908, where the 
positional parameters of a camera arrangement is provided), therefore Swift indicates 
camera parameters that describes the arrangement to enable the image acquisition. 

The applicant argues on pg. 20 3 rd H lines 1-6 of the remarks that, in regards to 
claim 7, Swift nor Osaka references teach or suggest a file extension which indicates 
which of a plurality of different three-dimensional display methods is used to display said 
three-dimensional image data. However, Osaka not only provide an extension which 
describes whether three-dimensional image data is displayed, but also indicates the 
manner in which to display the three-dimensional image data, such as in a checkered 
pattern (col. 1 7 lines 26-4 1 : ". . .whether or not a three-dimensional image is to be 
displayed has been recorded as information in the header 51 and... may be executed by a 
file extension... If it is determined at step S64 that this window has a three-dimensional 
image file... display driver 6 is controlled to present the three-dimensional display... on 
the basis of the three-dimensional image data obtained from the three-dimensional image 
data 52, the screen controller 9 controls the image painting unit 7 and the checkered 
mask-pattern painting unit 8 and causes a three-dimensional display to be presented at 
the position of the window of the stereoscopic display 12"). 

The applicant argues on pg. 20 4 th H line 1 - pg. 21 1 st \ line of the remarks that 
Swift nor Osaka teach the limitations of claims 10, 18 and 25, however Swift and 
Osaka's teaching of the limitations of claims 10, 18 and 25 are provided in the above 
office action, therefore the 35 U.S.C. 103(a) rejection of claims 10, 18 and 25 has been 
maintained. 
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The applicant argues on pg. 21 3 rd U lines 6-14 of the remarks that Swift does not 
teach or suggest including information in a parameter regarding any number of 
viewpoints other than the inherent left and right channel, and one of ordinary skill in the 
art would not have sufficient rationale to modify Swift to encode a parameter indicating 
the number of parallaxes in a first direction and/or a second direction into the multimedia 
file. However, Swift teaches including information regarding left and right channel 
parameters, as admitted by the applicant on pg. 21 3 rd lines 6-7 of the remarks, which 
thereby indicates first and second parallax directions for the left and right eyes of the 
observer. 

The applicant argues on pg. 22 4 th ^ lines 1-9 of the remarks that Swift teaches a 
"tag" for displaying three-dimensional image data and provides no mention of the 
arrangements of cameras se dot pick up the three-dimensional data. However, Swift 
provides teaching of the a plurality of predetermined camera arrangements of a plurality 
of camera which has picked up said three-dimensional image data (col. 10 lines 62-63: 
". . .right 1902 and left 1904 camera views of an object 1900." and is shown in Fig. 19: 
1906, 1908), where the arrangement of a plurality of cameras, as shown in Fig. 19, is 
thereby provided. 

The applicant argues on pg. 23 1 st U lines 6-8 of the remarks that Swift does not 
include information regarding the interval between adjacent cameras of the plurality of 
cameras which acquired the image data. However, Swift provides data describing the 
interval between adjacent cameras of the plurality of cameras which acquired the image 
data (col. 10 lines 52-65: ". ..embodiment allows... image views to be generated to 
simulate a. ..camera. ..and. ..can be used to simulate a wider stereoscopic camera..." and 
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is shown in Fig. 19), wherein the distance between each given camera position is set to 
form the camera image acquisition arrangement of Fig. 19. 

The applicant argues on pg. 23 2 nd U lines 8-10 of the remarks that in regards to 
claims 11,19 and 26, Uomori fails to teach or suggest that information regarding a 
distance to a convergence point formed by the view axes of a plurality of cameras which 
picked-up three-dimensional image data is encoded as a parameter and included in a 
multimedia file. However, Uomori provides information regarding a distance to a 
convergence point formed by the view axes of a plurality of cameras which picked-up 
three-dimensional image data (col. 5 lines 24-31: "...the fusional range verification 
section 11 calculates... the camera parameters... distance dx from camera converging 
point to imaginary camera position V. . .", and is shown in Fig. 2: dx, where the view axes 
of the plurality of camera positions converge to the convergence point), in which Swift 
teaches encoding camera parameters in a multimedia file (col. 4 lines 6-11: "The 
encoding processes used include independent compression of the Left and Right images. 
Independent compression of Left and Right provides better quality display 
output... methods include a video setup format for compression. ..Several tools are 
used... These include... a stereo compression analysis tool to detect if the stereo 
information is preserved after compression.'" and in col. 1 1 lines 12-16: "If too much 
compression is applied, the image will loose its stereoscopic impact. This embodiment 
provides a measure of the quality of the stereoscopic image that can be used to readjust 
the compression system", where a parameter, or stereo information, that provides 
information regarding how to correctly display and align the stereoscopic images form 
corresponding left/right points of view, is encoded in the file), in which it would have 
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been obvious to one of ordinary skill in the art at the time of invention to modify the 
encoded camera parameters of Swift with the convergence point distance calculation of 
Uomori because this modification would provide efficient storage of camera parameter 
information within one multimedia file, thereby ensuring visually appealing stereoscopic 
images are provided through accurate storage of the camera parallax and viewpoints of 
the image acquisition devices with respect to calculated distances from the device to a 
convergence point to provide efficient stereoscopic imagery to an observer. 



Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to SAID BROOME whose telephone number is (571)272- 
293 1 . The examiner can normally be reached on M-F 8:30am-5pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ulka Chauhan can be reached on (571)272-7782. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. Status 
information for unpublished applications is available through Private PAIR only. For 
more information about the PAIR system, see http://pair-direct.uspto.gov. Should you 
have questions on access to the Private PAIR system, contact the Electronic Business 
Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO 
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Customer Service Representative or access to the automated information system, call 
800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Said Broome/ 
Examiner, Art Unit 2628 

/Sajous, Wesner/ 

Primary Examiner, Art Unit 2628 



